New measures for evaluating creativity in scientific publications

The main objective of our ongoing research work is understanding the group dynamics of the scientific creative process, including the flow of research ideas, the creation of new ideas, and their impact on future ideas in collaborative communities. The principal means for pursuing this objective is by analyzing the main outputs of research work, such as scientific publications and their citations. The list of citations referenced in a publication gives an indication of the main sources of information related to and used by the paper. But, by itself, arguably, it does not inform which citations influenced the main contributions of the paper and how. Our underlying assumption is that creative ideas are novel and useful combinations of old ideas or knowledge [Amabile 1983]. Thus, understanding which citations have a direct contribution to a paper’s main novel ideas is critical as well as understanding which papers produce truly creative ideas (i.e. ideas that are both novel and useful). In order to understand how new ideas form current ideas and then propagate through a research community, a novel analysis methodology of the creative contribution of each citation is needed. The number of citations is considered to be by and large the main way of measuring a paper’s influence [Wanga et al. 2013]. More recently, citations have been used to gain insight about the dynamics of a research domain, including the temporal patterns in which papers are cited [Wanga et al. 2013], models to predict the total number of citations and the impact factor of journals [Wanga et al. 2013], the relation between creativity and co-occurence of citations [Uzzi et al. 2013], and the main path of a scientific domain’s evolution [Lucio-Arias and Leydersdorff 2008]. However, a common characteristics of current work is that citations are used as the single, main source of information for modeling scientific impact in large communities. There is little meaning assigned to a citation, e.g., its purpose or context of use in a paper, even though this knowledge is important in assessing the transformation and evolution of creative ideas within a research community. Citations are usually not correlated with the content of the paper, hence it is difficult to distinguish between citations that give a general reference to a domain or problem (such citations can arguably be replaced by other references) and citations in which the referred idea is the starting (triggering) point in the construction of a new solution. Second, research work is likely to be cited multiple times over time by the same group. Such groups usually work on a similar problem and/or have transformed the original idea for new applications. This insight helps understanding the importance of the cited paper for the specific community as well as the cited idea’s flexibility in being transformed to tackle new applications. Analyzing clusters of citations by the same research group (rather than individual citations) offers new insight about the usefulness of cited work and its importance in spawning new research. This presentation discusses two possible metrics for estimating the creativity of a scientific publication. Citations are clustered depending on their originating group (e.g., research laboratory) and classified into seven categories based on their purpose. This information is then used to evaluate the uniqueness (novelty) and usefulness of the presented contribution with respect to related work within